Skip to content

docs: large Wasm guide#99

Merged
marc0olo merged 2 commits into
mainfrom
docs/guides-canister-management-large-wasm
Apr 16, 2026
Merged

docs: large Wasm guide#99
marc0olo merged 2 commits into
mainfrom
docs/guides-canister-management-large-wasm

Conversation

@marc0olo
Copy link
Copy Markdown
Member

Summary

  • Why Wasm modules grow large (dependencies, embedded data, debug symbols)
  • gzip compression: recipe-based (shrink + compress in icp.yaml) and manual ic-wasm + gzip
  • Wasm chunk store: upload_chunk/install_chunked_code, automatic icp-cli chunking, cycle costs
  • Wasm64: wasm64-unknown-unknown target (6 GiB), build.sh and icp.yaml config from backend_wasm64 example
  • WebAssembly SIMD: auto-vectorization via .cargo/config.toml, per-function via target_feature, intrinsics
  • Troubleshooting: four common failure modes

Sync recommendation

informed by dfinity/portal — docs/building-apps/canister-management/compile.mdx, simd.mdx

@marc0olo
Copy link
Copy Markdown
Member Author

Review: Large Wasm Modules

Must fix

  • Broken internal link — lifecycle.md: The "Next steps" section links to lifecycle.md, but the actual file on disk is lifecycle.mdx. Per review guidelines, ls lifecycle.md fails and ls lifecycle.mdx succeeds. Astro resolves .md links to .mdx files at build time, so this will likely work at runtime — but it should be flagged for confirmation. The link [Canister lifecycle](lifecycle.md) resolves at build time if Astro's Starlight handles this transparently, which should be verified with a build run.

  • Inaccurate shrink description in recipe section: The page states "Setting shrink: true first removes unused functions and debug metadata." However, the actual Rust recipe template (icp-cli-recipes/recipes/rust/recipe.hbs) runs ic-wasm ... shrink --keep-name-section, which preserves the name section (function names). The name section is a form of debug metadata; describing shrink as removing "debug metadata" without the keep-name-section caveat is imprecise and internally inconsistent with the custom build script section (which correctly explains --keep-name-section). Suggested fix: "Setting shrink: true first removes unused functions and debug info while preserving function names for readable backtraces, then compress: true gzip-compresses the result."

Suggestions

  • Wasm64 memory limit clarification: The page says wasm64 supports "up to 6 GiB of addressable memory." This is confirmed by the portal (canisters.mdx states the Wasm memory maximum is 6 GiB). However, the wasm64 address space is technically 2^64 bytes — 6 GiB is an ICP platform-imposed limit, not the Wasm64 spec limit. A small clarification would improve accuracy: "ICP supports wasm64-unknown-unknown with up to 6 GiB of addressable heap memory (an ICP platform limit)."

  • Compression reduction percentage: "Gzip compression typically reduces Wasm binary size by 50-70%" — this specific range is not sourced from .sources/. The portal and recipe docs do not state a percentage. It is a reasonable engineering estimate, but consider softening to "significantly" or citing the basis for the estimate.

  • icp canister install automatic chunking claim: The page states "When you run icp deploy or icp canister install with a Wasm module larger than 2 MiB, icp-cli automatically uses the chunk store." The icp-cli reference docs do not explicitly document this automatic behavior. The claim is plausible and fits the tool's design intent, but it is not directly verifiable from .sources/icp-cli/docs/. The troubleshooting note "automatic chunk store support was added in v0.2.x" is similarly unverifiable from available sources — recommend a <\!-- TODO: verify output --> flag or confirmation from the icp-cli team.

  • CHUNK_STORE_SIZE link opportunity: The page mentions CHUNK_STORE_SIZE in the IC interface spec. Consider linking to the management canister reference page (../../reference/management-canister.md) or noting that the exact value is platform-defined, to help readers find the authoritative source.

Verified

  • Gzip magic bytes [0x1f, 0x8b, 0x08] — Confirmed against IC interface spec, which states: "If the module starts with byte sequence [0x1f, 0x8b, 0x08], then the system decompresses the contents."
  • compress and shrink recipe flags — Confirmed as real parameters in icp-cli-recipes/recipes/rust/recipe.hbs and recipes/rust/README.md.
  • gzip --no-name ... && mv *.gz pattern — Confirmed as the exact steps used in the Rust recipe template.
  • Chunk store API methods (upload_chunk, install_chunked_code, stored_chunks, clear_chunk_store) — All confirmed against IC interface spec.
  • 1 MiB chunk size limit and CHUNK_STORE_SIZE — Confirmed: IC spec states "The size of each chunk must be at most 1MiB" and references CHUNK_STORE_SIZE.
  • ic_cdk::api::instruction_counter() — Confirmed in cdk-rs/ic-cdk/src/api.rs (wraps performance_counter(0)).
  • SIMD "more than 200 vector instructions" — Confirmed: portal simd.mdx states exactly this.
  • .cargo/config.toml SIMD global config — Confirmed against portal simd.mdx.
  • #[target_feature(enable = "simd128")] before #[ic_cdk::query] — Confirmed in examples/rust/simd/mat_mat_mul/src/lib.rs (lines 111-112).
  • core::arch::wasm32 for SIMD intrinsics — Confirmed by portal SIMD docs and examples.
  • backend_wasm64 build.sh — Page script matches examples/rust/backend_wasm64/build.sh (simplified but accurate).
  • icp.yaml for wasm64 — Matches examples/rust/backend_wasm64/icp.yaml exactly.
  • wasm_memory_limit: 4gib format — Confirmed valid in icp-cli/docs/reference/canister-settings.md.
  • icp deploy and icp canister install — Both confirmed in .sources/icp-cli/docs/reference/cli.md.
  • optimization.md link — File docs/guides/canister-management/optimization.md exists.
  • execution-errors.md link — File docs/reference/execution-errors.md exists.
  • No dfx references — None found.
  • Frontmatter — Complete and consistent (title, description, sidebar.order).
  • Upstream comment — Present and correctly formatted.
  • File extension.md (not .mdx), correct for a page with no interactive components.

…ccuracy

- Fix shrink: true description to note --keep-name-section preserves function names
- Soften unsourced 50-70% compression estimate to "significantly"
- Clarify 6 GiB is an ICP platform limit, not a Wasm64 architectural limit
- Add TODO flag for unverifiable automatic chunking behavior claim
- Add management-canister reference link for CHUNK_STORE_SIZE
@marc0olo
Copy link
Copy Markdown
Member Author

<!-- feedback-addressed -->
Feedback addressed:

Changes applied

  • Fixed shrink: true description (Must fix): Updated "removes unused functions and debug metadata" to "removes unused functions and debug info while preserving function names for readable backtraces". Verified against .sources/icp-cli-recipes/recipes/rust/recipe.hbs which runs ic-wasm ... shrink --keep-name-section, confirming the name section is preserved. The original description was internally inconsistent with the custom build script section that correctly explains --keep-name-section.

  • Softened unsourced compression percentage (Suggestion): Changed "50-70%" to "significantly". The specific range was not found in .sources/portal/ or .sources/icp-cli-recipes/ docs. The new wording is accurate without making an unverifiable claim.

  • Clarified Wasm64 6 GiB as an ICP platform limit (Suggestion): Updated "up to 6 GiB of addressable memory" to "up to 6 GiB of addressable heap memory (an ICP platform limit)". Verified against .sources/portal/docs/building-apps/essentials/canisters.mdx which confirms this is the Wasm memory maximum on ICP — not a Wasm64 architectural constraint (which would be 2^64 bytes).

  • Added TODO flag for automatic chunking claim (Suggestion): Added a TODO comment to the sentence claiming icp-cli automatically uses the chunk store for modules >2 MiB. The icp-cli reference docs do not explicitly document this automatic behavior, making the claim unverifiable from available sources.

  • Added management-canister reference link for CHUNK_STORE_SIZE (Suggestion): Updated the chunk store bounded description to link to ../../reference/management-canister.md so readers can find the authoritative value. File existence verified with ls.

Items skipped

  • lifecycle.md link (Must fix — confirmed correct, no change needed): The reviewer flagged [Canister lifecycle](lifecycle.md) and noted lifecycle.mdx exists but lifecycle.md does not. Per CLAUDE.md convention: "links to .mdx pages always use .md extension (Astro resolves both)". The link lifecycle.md is the correct form per project convention. The reviewer's own text acknowledges "Astro's Starlight handles this transparently" — this is exactly the documented behavior. No change made.

@marc0olo marc0olo merged commit 6ef9252 into main Apr 16, 2026
1 check passed
@marc0olo marc0olo deleted the docs/guides-canister-management-large-wasm branch April 16, 2026 19:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant